Automated layouts optimized for multi-screen and multi-camera videoconferencing calls

ABSTRACT

A videoconferencing device that determines a spatial arrangement of the video streams output at a first endpoint based on a number of display devices being utilized at the first endpoint and a number of cameras being utilized at each of the one or more additional endpoints.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the filing date of U.S.Provisional Application No. 62/090,212, filed on Dec. 10, 2014, theentire contents of which are incorporated herein by reference.

BACKGROUND

Videoconferencing entails exchange of audio, video, and otherinformation between at least two participants. Generally, avideoconferencing endpoint at each participant location will include acamera for capturing images of the local participant and a displaydevice for displaying images of remote participants. Thevideoconferencing endpoint can also include additional display devicesfor displaying digital content. In scenarios where more than twoendpoints participate in a videoconferencing session, a multipointcontrol unit (MCU) can be used as a conference controlling entity. TheMCU and endpoints typically communicate over a communication network,the MCU receiving and transmitting video, audio, and data channels fromand to the endpoints.

Telepresence technologies provide enhanced videoconferencing experienceto participants so that the near end participants feel as if they arepresent in the same room as the far end participants. Telepresencevideoconferencing can be provided for various conferencing systems,ranging from two person point-to-point videoconferencing systems tomulti-participant multipoint videoconferencing systems. Typically,telepresence utilizes multiple cameras to capture images of near endparticipants and multiple displays to display images of far endparticipants. Multiple video streams are transmitted from multipleendpoints to the MCU to be combined into one or more combined videostreams that are sent back to the endpoints to be displayed on multipledisplay devices. For example, in a telepresence system involving threeendpoints, each endpoint having three cameras, the MCU will receive ninevideo streams. The MCU will have to combine the nine received videostreams into one or more combined video streams, which are sent back tobe displayed on the display devices at each endpoint. These nine videostreams will have to be laid out for each endpoint based on the numberand type of displays at each endpoint. Furthermore, although the MCU mayreceive the information from the endpoint that the current speaker islocated at that endpoint, with more than one video stream being receivedfrom each endpoint the MCU may not be able to determine which one of themultiple video streams includes the current speaker. Thus, dynamicallyselecting one of many video streams received from an endpoint forprominent display may be difficult.

Commonly-owned U.S. Pat. No. 8,537,195, which is hereby incorporated byreference in its entirety, describes various techniques for assigningtelepresence streams to a display layout. However, even some embodimentsof such systems may not utilize all of the available screens to show theactive speaker and other participants in a mixed interactivetelepresence (“ITP”) call environment. Additionally, with current layoutmanagement tools, multi-screen environment administrators have a highupfront management task to coordinate layouts for end user environmentscenarios and these often fail to meet the desired speaker switchingneeds for the end users. For example, many current active speakerswitching embodiments prioritize sites in a call based on number ofcamera streams, which does not always factor in the active speaker orother key meeting analytics to optimize the user experience withautomated layouts. This leads to scenarios where the active speaker maynot be shown at all on screens at a particular location. Anotherundesirable scenario that can arise in multi-screen environments is whenactive speaker locations move around so much that users are disorientedand unsure of where to focus.

In some currently available embodiments, conference rooms with multiplemonitors may locate the main speaker on a single monitor, usually in thecenter, with other participants being shown in a filmstrip view at thebottom. Various embodiments of a film strip arrangement, includingdynamic assignment of users to the various view positions, are describedin Provisional U.S. Patent Application 62/002,561, filed May 23, 2014and entitled, “Method And System For New Layout Experience In VideoCommunication,” which is hereby incorporated by reference in itsentirety.

In some variations of such an arrangement, if the speaker is a singlecamera site and the conference room viewing the speaker has threemonitors, the speaker might show up full screen on the center monitor,while other participants would show on the left and right monitors asfilm strips at the bottom of a mostly black screen.

Other conventional videoconferencing arrangements reposition videostreams based on the location of the current speaker. These arrangementscan be unnecessarily jarring to viewers, especially when endpoints areutilizing different numbers of cameras and outputting different numbersof video streams.

Therefore, in order to overcome this problem arising in the realm ofvideo conferencing, there is a need for rule-based systems forcontrolling video layouts in multi-site, multi-camera videoconferencing.

SUMMARY

Two new videoconference layout modes are provided: A speaker prioritymode, which may be used to ensure that video streams from an endpointthat includes a current speaker are displayed sufficiently prominently,and a participant priority mode, which can be used when it is desirablefor all participants in a conference to be displayed as effectively aspossible.

The speaker priority mode may include one or more reserved screens thatinclude primary display areas allocated for current speakers. The numberof reserved screens output at an endpoint may be based on a number ofdisplay devices being utilized at the endpoint and a number of camerasbeing utilized at each of the other endpoints.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an exemplary multipoint videoconferencing system.

FIG. 2 shows an exemplary videoconferencing endpoint.

FIG. 3 shows an exemplary videoconferencing system.

FIG. 4A-4D show exemplary reserved screens.

FIG. 5 show exemplary grid screens.

FIGS. 6A-6C show exemplary layouts for endpoints utilizing two displaydevices when none of the other endpoints are utilizing four cameras.

FIGS. 7A-7M show exemplary layouts for endpoints utilizing two displaydevices when at least one of the other endpoints is utilizing fourcameras.

FIGS. 8A and 8B show exemplary layouts for endpoints utilizing threedisplay devices when none of the other endpoints are utilizing three orfour cameras.

FIGS. 9A-9C show exemplary layouts for endpoints utilizing three displaydevices when at least one of the endpoints is utilizing four cameras(and none of the other endpoints are utilizing three cameras).

FIGS. 10A-10D show exemplary layouts for endpoints utilizing threedisplay devices when at least one of the other endpoints is utilizingthree cameras.

FIG. 11 is a flowchart showing an exemplary process to determine thenumber of reserved screens and the number of grid screens at anendpoint.

DETAILED DESCRIPTION

FIG. 1 depicts an exemplary multipoint videoconferencing system 100.System 100 can include network 110, one or more multipoint control units(MCU) 106, and a plurality of endpoints A-E 101-105. Network 110 can be,but is not limited to, a packet switched network, a circuit switchednetwork, or a combination of the two. Endpoints A-E 101-105 may send andreceive both audio and video data. Communications over the network canbe based on communication protocols such as H.320, H.324, H.323, SIP,etc., and may use compression standards such as H.263, H.264, etc. MCU106 can initiate and manage videoconferencing sessions between two ormore endpoints. Generally, MCU 106 can mix audio data received from oneor more endpoints, generate mixed audio data, and send mixed audio datato appropriate endpoints. Additionally, MCU 106 can receive videostreams from one or more endpoints. One or more of these video streamsmay be combined by the MCU 106 into combined video streams. Videostreams, combined or otherwise, may be sent by the MCU 106 toappropriate endpoints to be displayed on their respective displayscreens. As an alternative, MCU 106 can be located at any one of theendpoints A-E 101-105.

FIG. 2 shows an exemplary endpoint 200 with multiple cameras andmultiple display devices. Cameras 202-205 capture images of the localparticipants present in the conference room, and can be arranged in arow to capture different portions of the conference room. Accordingly,cameras 202-205 can be labeled FR (far-right), CR (center right), CL(center left), and FL (far left). Of course, these labels are onlyexemplary. Different labels can also be used, for example, if camera 205is used to capture images of all participants then it can be labeled asWR (whole room). In another instance, camera 202 can be a pan-tilt-zoom(PZT) type camera that captures the image of the current speaker only,from among the local participants, and thus can be labeled CS (currentspeaker). Labels or attributes can be associated with the cameras andstored in memory in the processor 209.

Processor 209 can include a codec 210 for compressing and decompressingvideo and audio streams. For example, codec 210 can compress videostreams generated by cameras 202-205 to generate compressed videostreams, which can be transmitted to remote endpoints and/or an MCU.Additionally, codec 210 can decompress video streams received from theremote endpoints or the MCU, and display the video streams on displaydevices 206, 207, and 208. Codec 210 can include video codecs such asH.261 FCIF, H.263 QCIF, H.263 FCIF, H.261 QCIF, H.263 SQCIF, H.264,etc., and audio codecs such as G.711, G.722, G.722.1, G.723.1, etc.

Processor 209 can communicate with a speaker locator module 213, whichdetermines the location of the current speaker, i.e., the participantthat is currently speaking. Information provided by speaker locatormodule 213 can be used by processor 209 to determine which of thereceived video streams includes the current speaker. Speaker locatormodule 213 can employ a microphone array 224 that analyzes the soundreceived from a source, such as the current speaker, to determine thelocation of the current speaker with respect to the microphone array224. The microphone array 224 can include a series of spaced microphonesthat can be arranged horizontally, vertically, or in combination.Typically, at least one of the microphones in the array can be assignedas a reference microphone. A number of candidate locations can bepredetermined where the distance of the candidate locations from each ofthe microphones is known. Acoustic signals captured by each of themicrophones can be delayed with respect to the acoustic signal capturedby the reference microphone. This delay can be, in part, a function of acandidate source location and microphone location with respect to thereference microphone. Signal energies of each of the delayed signalsassociated with each candidate location can then be determined.Subsequently, the candidate location associated with the highest signalenergy can be selected as the location that best estimates the actuallocation of the audio source. In other words, using maximum likelihoodestimation, a predetermined candidate source that is likely to be thebest estimate of the actual location of the audio source can be selectedas the location of the audio source. Clearly, the accuracy of theestimation can improve with an increase in the number and spatialdistribution of the candidate locations. For example, 61 candidatelocations can be used at an approximate radial distance of 10 feet fromthe microphones. More details on determining locations of participantsusing microphone arrays are disclosed in commonly assigned U.S. Pat. No.6,912,178 entitled “System and method for computing a location of anacoustic source,” by Chu et al., and is hereby incorporated byreference.

Typically, the spatial relationship between the microphone array 224 andthe cameras 202-205 remains fixed. Therefore, location of the currentspeaker known with respect to the microphone array 224 can be readilytransformed into the location of the current speaker with respect to thecameras simply by changing the frame of reference. Each camera, with itsparticular pan-zoom-tilt settings, can capture a particular portion ofthe conference room, the boundaries of which portion can bepredetermined. Thus, the processor 209 can determine if the location ofthe current speaker, as expressed within the reference frame of thecamera, lies within the portion of the conference room captured by thatcamera. If the current speaker is located within the portion of theconference room captured by a camera, processor 209 can instruct thestream attribute module 212 to assign the attribute “Speaker” to thevideo stream generated by that camera.

Stream attribute module 212 can assign attributes to the outgoingstreams. These attributes can qualify the outgoing video stream in amanner that is useful for the MCU and/or the remote endpoint forrendering and displaying the video stream. These attributes can be addedto outgoing streams during transmission. For example, protocols such asH.323 and H.320 can be extended to include attribute definitions thatmay be used to label outgoing data streams. Video streams can havevarious attributes. For example, video streams can have positionalattributes that identify the relative location of the camera that is thesource of that video stream. As shown in FIG. 2, outgoing streams215-218 can have attributes that indicate the relative position of theirsource cameras. Stream 215, for example, is labeled “FR” because it issourced from the camera 202, which is placed in the far right (FR)position. Similarly, video stream 217 is labeled “CL” because its sourcecamera 204 is in a center left (CL) position.

Video streams can also have role based attributes such as “people” and“content.” Video streams can be labeled with “people” attribute if thevideo streams include images of people/participants. Role basedattributes can further have hierarchical classification. For example,where a number of participants in a video conference take turnspresenting information, classification of “people/presenter” and“people/audience” may be provided. A “people/presenter” attribute canindicate that the associated video stream includes images of people thatare to be prominently displayed irrespective of whether the video streamincludes a speaker or not. Video streams that contain digital content,such as presentations, can have a “content” attribute. For example,processor 209 receives data content from computer 224, which datacontent can include presentations, documents, videos, etc. Data contentcan be compressed and given the attribute “CON.” The endpoint can alsoinclude a dedicated data content display, which displays data streamsreceived from MCU or remote endpoints.

Video streams can also have more than one attribute. For example, avideo stream can have both role based and “Speaker” attribute, such as“people/audience, Speaker,” “people/presenter, Speaker,” etc. The“Speaker” attribute can be assigned independent of the role of the videostream. For example, even if the current speaker is included in a videostream having the “people/audience” role, a “Speaker” attribute can beadditionally assigned to that video stream. As another example, videostreams can have both positional and “Speaker” attribute, such as, videostream 215—where in addition to having the “FR” attribute it can alsohave the “Speaker” attribute. As previously described, processor 209 candetermine which camera is capturing the current speaker. The processorcan then add the “Speaker” attribute to the video stream generated bythat camera. In the example shown in FIG. 2, the current speaker isbeing captured by camera 202; therefore, video stream 215 has theattribute “Speaker” associated with it. If the location of the currentspeaker changes, then the processor can reassign the “Speaker” attributeto the video stream that currently includes the current speaker. Forexample, if a different speaker begins to talk, and the image of thatcurrent speaker is captured by camera 204, then video stream 217 will beassigned the attribute “Speaker” in addition to the “CL” attribute.

Although only a single outgoing stream 215 of FIG. 2 is shown to havemultiple attributes, such multiple attributes can be assigned to morethan one stream. For example, stream 218 also can have multipleattributes such as “people/presenter” and “FL” assigned to it. Thus,video streams can be assigned one or more attributes, and the assignedattributes can be role based, camera position based, current speakerbased, or based on any other selected property. It is understood thatthe attributes shown in FIG. 2 can be encoded for transmission.

Video streams, data streams, and audio streams, along with theirattributes can be transmitted by the endpoint 200 to an MCU or a remoteendpoint via network 110. A transmitter/receiver 214 can serve as aphysical interface between the endpoint 200 and the network 110. Tx/Rx214 can also receive video streams from the MCU or remote endpoints. Forexample, video streams 219-221 are received by the processor 209. Videostreams 219-221 can include attributes that qualify the video streamsand can be used by the processor 209 for rendering or reproducing thevideo streams. For example, video stream 219 has attributes “R” and“Speaker.” Processor 209 can display video stream 219 on display device208, which is positioned to the right. Furthermore, because video stream219 has the “Speaker” attribute as well, the video stream may bedisplayed with more prominence. Likewise, video stream 220 withattribute “C” can be displayed on display device 207 and video stream221 with attribute “L” can be displayed on display device 206.

FIG. 3 shows an exemplary conferencing system 300 in which two or moreendpoints communicate with each other via an MCU 106. Endpoints 101,102, and 103 can be similar to the exemplary endpoint shown in FIG. 2.Each endpoint can have various configurations of cameras and displayscreens for providing a telepresence experience to their respectiveparticipants. For example, endpoint A 101 can include 4 cameras and 4display devices, endpoint B 102 can include 4 cameras and 3 displaydevices, while endpoint C 103 can include 1 camera and 2 displaydevices. Video streams can be exchanged between each endpoint and theMCU 106. For clarity, only video streams to and from endpoint A 101 havebeen shown in detail. MCU 106 can also include network interface 328,via which video, audio, and signaling data can be sent and received.

MCU 106 includes memory 330 and one or more hardware processors, such asa processor 340 and a signal processor 350. The memory 330 may includeany tangible computer-readable storage medium configured to storeinstructions that, when executed by the one or more processors 340 and350, cause the MCU 106 to perform the process steps described herein.The memory 330 may include, for example, read-only memory (ROM),random-access memory (RAM), magnetic disc storage media, optical storagemedia, solid state (e.g., flash) memory, etc. The processor 340 may beany hardware device configured to carry out instructions stored in thememory 330 by performing the arithmetic, logical, control, andinput/output (I/O) operations specified by those instructions. Theprocessor 340 may include, for example, the Intel Core®, Pentium® andCeleron® processor families from Intel, the Cortex and ARM processorfamilies from ARM, etc. (INTEL CORE, PENTIUM and CELERON are registeredtrademarks of the Intel Corporation. CORTEX is a registered trademark ofthe ARM Limited Corporation. ARM is a registered trademark of the ARMLimited Company.) The signal processor 350 may be any hardware deviceconfigured to receive, arrange, and output video streams as describedherein. The processor 340 and the signal processor 350 may be integratedin a single chip, for example the TMS320DM6467 or TMS320DM6468, whichare provided by Texas Instruments Corporation. The processor 340 and/orsignal processor 350 may also include internal memory including (forexample, cache memory).

MCU 106 can include a layout manager 302 and a mixer 303. The layoutmanager 302 can determine display or video layouts, which include anarrangement of video streams sent to each endpoint. In determining videolayouts, the layout manager 302 not only determines which ones of thereceived video streams are to be sent to an endpoint, but also thespatial arrangement in which they are to be displayed. Thisdetermination can be based on the attributes associated with thereceived video streams and configuration information associated with theendpoint. The layout manager 302 can determine attributes associatedwith each video stream received by MCU 106. For example, attributes “FR,Speaker” 317, “CR” 318, “CL” 319, and “FL” 320 associated with videostreams 307, 308, 309, and 310 can be received from endpoint A 101.Similarly, video streams and their attributes can also be received fromendpoints B 102 and C 103 (denoted, for simplicity, by 315 and 316,respectively), and any additional endpoints. Configuration information329 received from each endpoint can include number of display devices,aspect ratio and resolution of display devices, existence of a dedicatedcurrent speaker display device, type of encoding used, etc. As will bedescribed with further detail below, the layout manager 302 can generatearrangement of video streams sent to each endpoint. This arrangement iscommunicated to the mixer 303 for execution by way of signal path 321.

Mixer 303 can receive video streams from one or more endpoints. Mixer303 can execute the arrangement of video streams determined by thelayout manager 302. For example, mixer 303 can receive video streamsfrom endpoints A 101, B 102, and C 103, combine the video streams basedon signals 321 received from the layout manager 302, and send combinedvideo streams back to each endpoint. Mixer 303 can include a codec 322for decoding incoming video and audio streams and encoding outgoingvideo and audio streams. For example, audio codecs can include standardcodecs such as, G.711, G.722, G.722.1, G.723.1, etc. Video codecs caninclude standard codecs, such as, H.261 FCIF, H.263 QCIF, H.263 FCIF,H.261 QCIF, H.263 SQCIF, H.264, etc. Codec 322 can also change theencoding scheme of outgoing audio and video streams based on theencoding scheme used at the receiving endpoint.

Commonly, the layout manager 302 is a program stored in the memory 330and executed on the processor 340 while the mixer 303 is a programstored in the memory 330 and executed on the signal processor 340 (ormultiple signal processors 340). Mixer 303 can also include an imageprocessing module 325 for carrying out manipulation of video framesreceived from various endpoints. Such manipulations can includecombining two or more video frames into one frame, scaling, cropping,overlaying, etc., more details of which are disclosed in commonlyassigned U.S. patent application Ser. No. 12/581,626 entitled “Systemand method for combining a plurality of video stream generated in avideoconference,” by Avishay Halavy, and is hereby incorporated byreference.

Mixer 303 can also include a stream attribute module 327 for assigningattributes to outgoing streams. For example, stream attribute module 327can assign attributes “FL”, “CL”, “CR, Speaker”, and “FR, CON” tostreams 311, 312, 313, and 314, respectively. The stream attributemodule 327 can receive instructions on which attributes to assign toparticular outgoing video streams from the layout manager 302.

MCU 106 can be compliant with the ITU standards, such as, but notlimited to, H.320, H.323, and H.324. Accordingly, the processor 340 canbe part of a media controller (MC), while the signal processor 350 canbe part of a media processor (MP). Mixer 303 can be implemented onapplication specific integrated circuits (ASICs), microcontrollers,FPGAs, hardware/firmware combination, software running onmicroprocessors, etc. Various modules within the mixer 303, e.g., codec322, Image processing module 325, and stream attribute module 327 can beindividual hardware modules, firmware modules, a software moduleexecuted by a processor (e.g., the signal processor 350), etc. Layoutmanager 302 can also be implemented separately as a hardware componentsuch as a microcontroller, ASIC, FPGA, hardware/firmware combination, asoftware module executed by a processor (e.g., the processor 340), etc.

In some embodiments, each of the endpoints 101-105 may include a layoutmanager 302 and a mixer 303 for determining the arrangement of the videostreams displayed at that endpoint.

In some embodiments, the endpoints 101-105 may communicate without anMCU 106. Accordingly, some or all of the features shown in FIG. 3 asincorporated within the MCU 106 (i.e., the processor 340, the layoutmanager 302, etc.) may be incorporated within one or more of theendpoints 101-105.

Having thus described the operating environment, attention will nowshift to the layout selections available according to the teachingherein. In addition to the conventional “continuous presence” layoutmode, which is known to those skilled in the art, two additional modesare available: “Speaker Priority” and “Participants Priority.” The“Speaker Priority” mode gives priority to video streams from theendpoint that includes the current speaker (the “current speakerendpoint”) and displays those video streams in a larger size that videostreams from other endpoints. The “Participants Priority” mode givespriority to displaying all participants possible. The telepresencelayout mode may be set by either a user or an administrator in aconference profile setting of either an endpoint or a MCU. As describedbelow, the layout selections are characterized as either reservedscreens or grid screens.

Speaker Priority Mode

FIG. 4A-4D show exemplary reserved screens 400.

As shown in FIG. 4A, the reserved screen 400 a includes a primarydisplay area 401 and a plurality of (e.g., four) secondary display areas420-423, which are smaller in size than the primary display area 401.The secondary display areas 420-423 may be arranged as filmstrip (asdescribed, for example, in Provisional U.S. Patent Application62/002,561, which is referenced above and incorporated by reference).The secondary display areas 420-423 may be above and/or below theprimary display area 401. As shown in FIG. 4B, the reserved screen 400 bincludes two primary display areas 402 and 403 and a plurality ofsecondary display areas 420-423 above and/or below the primary displayareas 402 and 403. As shown in FIG. 4C, the reserved screen 400 cincludes three primary display areas 404-406 and a plurality ofsecondary display areas 420-423. As shown in FIG. 4D, the reservedscreen 400 d includes four primary display areas 407-410 and a pluralityof secondary display areas 420-423.

If an endpoint is utilizing a single display device, the layout manager302 selects an appropriate layout for that display device based on thenumber of cameras at the endpoint that includes the current speaker (the“current speaker endpoint”). For example, if the current speakerendpoint includes only one camera, the layout manager 302 selects thereserved screen 400 a and outputs the video stream from the currentspeaker endpoint in the primary display area 401. Similarly, if thecurrent speaker endpoint has two cameras, the system selects thereserved screen 400 b and outputs the video streams from the currentspeaker endpoint in the primary display areas 402 and 403. If thecurrent speaker endpoint has three cameras, the layout manager 302selects the reserved screen 400 c and outputs the video streams from thecurrent speaker endpoint in the primary display areas 404-406. If thecurrent speaker endpoint has four cameras, the layout manager 302selects the reserved screen 400 d and outputs the video streams from thecurrent speaker endpoint in the primary display areas 407-410. In eachinstance, video streams from additional endpoints may be shown insecondary display areas 420-423.

FIG. 5 shows exemplary grid screens 500. The grid screens 500 includedisplay areas of substantially equal size arranged in columns and rows.The grid screens 500 may have an equal number of columns and rows. Forexample, the grid screen 500 a includes four display areas 501-504arranged in two rows and two columns. Similarly, the grid screen 500 bincludes nine display areas 511-519 arranged in three rows and threecolumns. The grid screen 500 c includes sixteen display areas 521-535arranged in four rows and four columns. As one of ordinary skill in theart will recognize, grid screens 500 may have any number of displayareas arranged in any number of rows and any number of columns(including one display area in a single row/column).

FIGS. 6A-6C and 7A-7M show exemplary layouts selected by the layoutmanager 302 for an endpoint utilizing two display devices. At anendpoint utilizing two display devices, the number of reserved screens400 selected by the layout manager 302 may be based on the number ofcameras being utilized at other endpoints in the system. If one of theother endpoints is utilizing four cameras, the layout manager selectstwo reserved screens 400. If none of the other endpoints are utilizingfour cameras, the layout manager 302 selects one reserved screen 400 andone grid screen 500.

FIGS. 6A-6C illustrate layouts selected by the layout manager 302 at anendpoint utilizing two display devices when none of the other endpointsare utilizing four cameras.

As shown in FIG. 6A, if the current speaker endpoint is utilizing onecamera, the layout manager 302 selects the reserved screen 400 a andoutputs the video stream from the current speaker endpoint in theprimary display area 401. As shown in FIG. 6B, if the current speakerendpoint is utilizing two cameras, the layout manager 302 selects thereserved screen 400 b and outputs the video streams from the currentspeaker endpoint in the primary display areas 402 and 403. As shown inFIG. 6C, if the current speaker endpoint is utilizing three cameras, thelayout manager 302 selects the reserved screen 400 c and outputs thevideo streams from the current speaker endpoint in the primary displayareas 404-406.

In each of the embodiments disclosed herein, the layout manager mayhighlight the video streams from the current speaker endpoint. Forexample, the video streams from the current speaker endpoint may beshown with a thicker or different color border than video streams fromother endpoints.

The layout manager 302 also outputs the video streams from the otherendpoints in the display areas of the grid screens and, if necessary,the secondary display areas 420-423. In order to minimize disruption anddisplay the video streams in the best possible layout, the layoutmanager 302 selects the grid screen 500 as follows:

If all of the video streams from the other endpoints can be displayed ina 3×3 (or smaller) grid screen, the secondary display areas 420-423 arenot used. If there are more than nine video streams from additionalendpoints, video streams are output to the secondary display areas420-423 in addition to the grid screen 500 b. If there are more thanthirteen video streams from additional endpoints, layout manager 302selects the 4×4 grid screen 500 c.

If a grid screen 500 includes multiple video streams from the sameendpoint, the layout manager 302 outputs the video streams from thatendpoint in the same row. Accordingly, the layout manager 302 determinesthe minimum grid size based on the maximum number of cameras in otherendpoints. For example, if one of the other endpoints is utilizing fourcameras, the minimum grid size is the 4×4 grid screen 500 c. Similarly,if one of the other endpoints is utilizing three cameras, the minimumgrid size is the 3×3 grid screen 500 b. If one of the other endpoints isutilizing two cameras, the minimum grid size is the 2×2 grid screen 500a.

Video streams are output to the secondary display areas 420-423 only ifvideo streams are being output to all of the display areas of all of thegrid screen 500. In other words, it is preferable to have fewer videostreams in the secondary display areas 420-423 than to have an emptydisplay area of a grid screen 500.

FIGS. 7A-7M show exemplary layouts selected by the layout manager 302for an endpoint utilizing two display devices when at least one of theother endpoints is utilizing four cameras.

As shown in FIG. 7A, if the current speaker endpoint is utilizing fourcameras, the layout manager selects the reserved screen 400 b for bothof the display devices and outputs the four video streams from thecurrent speaker endpoint in the primary display areas 402 and 403 ofboth display devices.

If the current speaker endpoint is utilizing less than four cameras, thevideo streams from the current speaker endpoint do not need to bedisplayed across both of the reserved screens 400. Because the currentspeaker endpoint may be displayed on a single display device,conventional systems may output the video streams on a single displaydevice and output the other video streams on the second display device(perhaps in a layout similar to a grid screen 500). Doing so, however,would cause all of the video streams to be moved and resized. Then, ifan endpoint utilizing four cameras were to become the current speakeragain, all the video streams would again need to be resized. Therefore,in order to overcome this problem arising in the realm of videoconferencing, the layout manager 302 continues to output two reservedscreens 400 regardless of whether the current speaker endpoint isutilizing four cameras. If the current speaker endpoint is utilizingfewer than four cameras, the layout manager 302 selects a secondreserved screen 400 based on the number of cameras being utilized at theendpoint that includes the previous speaker (“the previous speakerendpoint”)

As shown in FIG. 7B, if the current speaker endpoint is utilizing onecamera and the previous speaker endpoint is utilizing one camera, thelayout manager 302 selects the reserved screen 400 a for both of thedisplay devices, outputs the video stream from the current speakerendpoint in the primary display area 401 of one of the display devices(e.g., the left display device), and outputs the video stream from theprevious speaker endpoint in the primary display area 401 of the otherdisplay device (e.g., the right display device).

As shown in FIG. 7C, if the current speaker endpoint is utilizing onecamera and the previous speaker endpoint is utilizing two cameras, thelayout manager 302 selects the reserved screen 400 a and the reservedscreen 400 b, outputs the video stream from the current speaker endpointin the primary display area 401, and outputs the video streams from theprevious speaker endpoint in the primary display areas 402 and 403.

As shown in FIG. 7D, if the current speaker endpoint is utilizing onecamera and the previous speaker endpoint is utilizing three cameras, thelayout manager 302 selects the reserved screen 400 a and the reservedscreen 400 c, outputs the video stream from the current speaker endpointin the primary display area 401, and outputs the video streams from theprevious speaker endpoint in the primary display areas 404-406.

As shown in FIG. 7E, if the current speaker endpoint is utilizing onecamera and the previous speaker endpoint is utilizing four cameras, thelayout manager 302 selects the reserved screen 400 a and the reservedscreen 400 d, outputs the video stream from the current speaker endpointin the primary display area 401, and outputs the video streams from theprevious speaker endpoint in the primary display areas 407-410.

As shown in FIG. 7F, if the current speaker endpoint is utilizing twocameras and the previous speaker endpoint is utilizing one camera, thelayout manager 302 selects the reserved screen 400 b and the reservedscreen 400 a, outputs the video streams from the current speakerendpoint in the primary display areas 402 and 403, and outputs the videostream from the previous speaker endpoint in the primary display area401.

As shown in FIG. 7G, if both the current speaker endpoint and theprevious speaker endpoint are utilizing two cameras, the layout manager302 selects the reserved screens 400 b for both of the display devices,outputs the video stream from the current speaker endpoint in theprimary display areas 402 and 403 of one of the display devices (e.g.,the left display device), and outputs the video streams from theprevious speaker endpoint in the primary display areas 402 and 403 ofthe other display device (e.g., the right di splay device).

As shown in FIG. 7H, if the current speaker endpoint is utilizing twocameras and the previous speaker endpoint is utilizing three cameras,the layout manager 302 selects the reserved screen 400 b and thereserved screen 400 c, outputs the video streams from the currentspeaker endpoint in the primary display areas 402 and 403, and outputsthe video streams from the previous speaker endpoint in the primarydisplay areas 404-406.

As shown in FIG. 7I, if the current speaker endpoint is utilizing twocameras and the previous speaker endpoint is utilizing four cameras, thelayout manager 302 selects the reserved screen 400 b and the reservedscreen 400 d, outputs the video streams from the current speakerendpoint in the primary display areas 402 and 403, and outputs the videostreams from the previous speaker endpoint in the primary display areas407-410.

As shown in FIG. 7J, if the current speaker endpoint is utilizing threecameras and the previous speaker endpoint is utilizing one camera, thelayout manager 302 selects the reserved screen 400 c and the reservedscreen 400 a, outputs the video streams from the current speakerendpoint in the primary display areas 404-406, and outputs the videostream from the previous speaker endpoint in the primary display area401.

As shown in FIG. 7K, if the current speaker endpoint is utilizing threecameras and the previous speaker endpoint is utilizing two cameras, thelayout manager 302 selects the reserved screen 400 c and the reservedscreen 400 b, outputs the video streams from the current speakerendpoint in the primary display areas 404-406, and outputs the videostream from the previous speaker endpoint in the primary display areas402 and 403.

As shown in FIG. 7L, if both the current speaker endpoint and theprevious speaker endpoint are utilizing three cameras, the layoutmanager 302 selects the reserved screens 400 c for both of the displaydevices, outputs the video stream from the current speaker endpoint inthe primary display areas 404-406 of one of the display devices (e.g.,the left display device), and outputs the video streams from theprevious speaker endpoint in the primary display areas 404-406 of theother display device (e.g., the right display device).

As shown in FIG. 7M, if the current speaker endpoint is utilizing threecameras and the previous speaker endpoint is utilizing four cameras, thelayout manager 302 selects the reserved screen 400 c and the reservedscreen 400 d, outputs the video streams from the current speakerendpoint in the primary display areas 404-406, and outputs the videostream from the previous speaker endpoint in the primary display areas407-410.

FIGS. 8A-8B, 9A-9C, and 10A-10C show exemplary layouts selected by thelayout manager 302 for endpoints utilizing three display devices.

At an endpoint utilizing three display devices, the number of reservedscreens 400 selected by the layout manager 302 is based on the number ofcameras being utilized at other endpoints in the system. If one of otherendpoints in the system is utilizing three cameras, the layout manager302 selects three reserved screens 400. If one of the other endpoints inthe system is utilizing four cameras (and none of the other endpointsare utilizing three cameras), the layout manager selects two reservedscreens 400 and one grid screen 500. If none of the other endpoints areutilizing three or four cameras, the layout manager 302 selects onereserved screen 400 and two grid screens 500.

FIGS. 8A and 8B show exemplary layouts selected by the layout manager302 for endpoints utilizing three display devices when none of the otherendpoints are utilizing three or four cameras.

As shown in FIG. 8A, if the current speaker endpoint is utilizing onecamera, the layout manager 302 selects the reserved screen 400 a andoutputs the video stream from the current speaker endpoint in theprimary display area 401. The reserved screen 400 a may be selected forthe center display device as shown.

As shown in FIG. 8B, if the current speaker endpoint is utilizing twocameras, the layout manager 302 selects the reserved screen 400 b forone of the display devices (e.g., for the center display device) andoutputs the video streams from the current speaker endpoint in theprimary display areas 402 and 403.

In each of the embodiments shown in FIGS. 8A-8B, the layout manager 302also selects the grid screens 500 that best display the additional videostreams from the additional endpoints. The process for selecting twogrid screens 500 is similar to the process for selecting a single gridscreen 500 as described above with reference to FIGS. 6A-6C. The layoutmanager 302 may select two grid screens 500 with different size grids.

FIGS. 9A-9C show exemplary layouts selected by the layout manager 302for endpoints utilizing three display devices when at least one of theendpoints is utilizing four cameras (and none of the other endpoints areutilizing three cameras). When at least one of the endpoints isutilizing four cameras (and none of the other endpoints are utilizingthree cameras), the layout manager 302 selects two reserved screens 400and one grid screen 500.

As shown in FIG. 9A, if the current speaker endpoint is utilizing fourcameras, the layout manager selects two reserved screens 400 b for twoof the display devices and outputs the video feeds from the currentspeaker endpoint in the primary display areas 402 and 403.

If the current speaker endpoint is not utilizing four cameras, the videostreams from the current speaker endpoint are not displayed across bothof the reserved screens 400. In order to minimize disruption for theviewers as described above, the layout manager 302 selects the reservedscreens 400 based on the number of cameras being utilized at both thecurrent speaker endpoint and the previous speaker endpoint.

As shown in FIG. 9B, for example, if the current speaker endpoint isutilizing one camera and the previous speaker endpoint is utilizing fourcameras, the layout manager 302 selects the reserved screen 400 a forone of the display devices (e.g., the center display device), outputsthe video stream from the current speaker in the primary display area401, selects the reserved screen 400 d as the other reserved screen, andoutputs the video streams from the previous speaker endpoint in theprimary display areas 407-410.

As shown in FIG. 9C, for example, if the current speaker endpoint isutilizing two cameras and the previous speaker endpoint is utilizing onecamera, the layout manager 302 selects the reserved screen 400 b for oneof the display devices (e.g., the center display device), outputs thevideo streams from the current speaker in the primary display areas 402and 403, selects the reserved screen 400 a as the other reserved screen,and outputs the video streams from the previous speaker endpoint in theprimary display area 401.

In each of the embodiments shown in FIGS. 9A-9C, the layout manager 302also selects the grid screen 500 that best displays the additional videostreams from the additional endpoints as described above with referenceto FIGS. 6A-6C.

FIGS. 10A-10C show exemplary layouts selected by the layout manager 302for endpoints utilizing three display devices when at least one of theother endpoints is utilizing three cameras. When one of the otherendpoints is utilizing three cameras, the layout manager 302 selectsthree reserved screens 400.

As shown in FIG. 10A, if the current speaker endpoint is utilizing threecameras, the layout manager selects the reserved screen 400 a for allthree display devices and outputs the video streams from the currentspeaker endpoint in the primary display areas 401.

If the current speaker endpoint is not utilizing three cameras, thevideo streams from the current speaker endpoint are not displayed acrossall three of the reserved screens 400. Accordingly, the layout manager302 may select the reserved screens 400 based on the number of camerasbeing utilized at both the current speaker endpoint and the previousspeaker endpoint.

As shown in FIG. 10B, if the current speaker endpoint is utilizing fourcameras, the layout manager 302 selects the reserved screen 400 b fortwo of the display devices and outputs the four video streams from thecurrent speaker endpoint in the primary display areas 402 and 403 ofboth of the reserved screens 400 b. The layout manager 302 also selectsthe third reserved screen 400 based on the number of cameras beingutilized in at the previous speaker endpoint. As shown in FIG. 10B, forexample, if the previous speaker endpoint is utilizing three cameras,the layout manager 302 selects the reserved screen 400 c and outputs thevideo streams from the previous speaker endpoint in the primary displayareas 404-406.

If the current speaker endpoint is utilizing one or two cameras, thelayout manager selects the appropriate reserved screen 400 for one ofthe display devices (e.g., the center display device) and selects theother two reserved screens 400 based on the number of cameras beingutilized at two previous speaker endpoints.

As shown in FIG. 10C, for example, if the current speaker endpoint isutilizing one camera, one previous speaker endpoint is utilizing fourcameras, and another previous speaker endpoint is utilizing two cameras,the layout manager selects the reserved screen 400 a and outputs thevideo stream from the current speaker endpoint in the primary displayarea 401, selects the reserved screen 400 d and outputs the four videostreams from one previous speaker endpoint in the primary display areas407-410, and selects the reserved screen 400 b and outputs the two videostreams from the other previous speaker endpoint in the primary displayareas 402 and 403.

In each of the embodiments disclosed herein, if a previous speakerendpoint is being displayed in one or more primary display area(s) andthe previous speaker endpoint becomes the current speaker endpoint(i.e., someone beings speaking again), the layout manager 302 may simplyhighlight the video streams from the previous speaker endpoint (now thecurrent speaker endpoint) instead of moving the video streams from theprevious speaker endpoint (now the current speaker endpoint) to thecenter display device.

As shown in FIGS. 10C and 10D, for example, if someone from the previousspeaker endpoint being shown in primary display areas 402 and 403 beginsspeaking, the layout manager 302 may highlight the display areas 402 and403 rather than moving the video streams from the endpoint being shownin the primary display areas 402 and 403 to the center display device.

At endpoints utilizing more than three display devices, the layoutmanager 302 selects the number of reserved screens 400 in the samemanner as for endpoints utilizing three display devices (except that,for each additional display device over three, the layout manager 302selects a grid screen 500 as discussed below.

FIG. 11 is a flowchart showing an exemplary process 1100 to determinethe number of reserved screens 400 and the number of grid screens 500 atan endpoint. The process 1100 may be performed by the layout manager302, which may be incorporated within the MCU 106 and/or an endpoint101-105 as described above.

If, at 1112, it is determined that the endpoint is utilizing one displaydevice, one reserved screen 400 is selected at 1114. If, at 1116, it isdetermined that the endpoint is utilizing two display devices, it isdetermined at 1118 whether any of the other endpoints are utilizing fourcameras. If so (1118: Yes), two reserved screens 400 are selected at1120. If none of the other endpoints are utilizing four cameras (1118:No), one reserved screen 400 and one grid screen 500 are selected at1122.

If it is determined that the endpoint is utilizing three or more displaydevices (1116: No), a determination is made at 1124 whether any of theother endpoints are utilizing three cameras. If so (1124: Yes), threereserved screens 400 are selected at 1126. If none of the otherendpoints are utilizing three cameras (1124: No), a determination ismade at 1128 whether any of the other endpoints are utilizing fourcameras. If so (1128: Yes), two reserved screens 400 and one grid screen500 are selected at 1130. If none of the other endpoints are utilizingfour cameras (1128: No), one reserved screen 400 and two grid screens500 are selected. At 1134, a determination is made whether there aremore than three display devices at the endpoint. If so (1134: Yes), anadditional grid screen 500 is selected for each additional displaydevice.

Participant Priority Mode

In general, participant priority mode can be used when it is desired toshow as many conference participants as possible using the largest cellsize possible. Such an arrangement outputs grid screens 500 to alldisplay devices, applying the same grid screen logic as in speakerpriority mode discussed above with reference to FIGS. 6A-6C.

If an endpoint is utilizing multiple cameras, the layout manager 302selects a grid screen 500 with sufficient columns to output all of thevideo streams from that endpoint in the same row.

If the layout manager 302 outputs multiple grid screens 500, the layoutmanager 302 attempts to output all of the video streams in grid screens500 with the same grid size. If there is an insufficient number ofdisplay areas, the layout manager 302 increases the number ofcolumns/rows of each of the grid screens by one. In other words, if two2×2 grid screens are insufficient and one 3×3 grid screen and one 2×2grid screen is insufficient the layout manager 302 determines whethertwo 3×3 grid screens are sufficient rather than using one 4×4 gridscreen and one 2×2 grid screen.

If the layout manager 302 outputs multiple grid screens 500 withdifferent grid sizes, the layout manager 302 outputs the grid screen 500with the smallest grid (and largest display areas) to acentrally-located display device and outputs video streams to a gridscreen 500 with a larger grid (and smaller display areas) only if thesmallest grid is full.

Rather than leaving a grid screen 500 blank, the layout manager 302partially fills more than one grid screen 500.

The layout manager 302 may highlight the video stream(s) from thecurrent speaker endpoint as described above.

Detailed descriptions of the embodiments described above are also shownin the attached appendices, wherein the endpoint that includes thecurrent speaker is sometimes referred to as the first priority and theendpoints that include the previous speakers are referred to as thesecond priority, the third priority, etc.

Appendix 1.1 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 in the speaker priority mode.

Appendix 1.1.1 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is one.

Appendix 1.1.2 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is two.

Appendix 1.1.3 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is three.

Appendix 1.1.4 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is four.

Appendix 1.2 shows exemplary grid screens 500 selected by the layoutmanager 302 in the speaker priority mode.

Appendix 1.2.1 shows one exemplary grid screens 500 selected by thelayout manager 302.

Appendix 1.2.2 shows two exemplary grid screens 500 selected by thelayout manager 302.

Appendix 1.2.3 shows three exemplary grid screens 500 selected by thelayout manager 302.

Appendix 1.3 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 in the speaker priority mode.

Appendix 1.3.1 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is three, there arethree reserved screens, the speaker location has one camera, the secondpriority location has three cameras and there are five additional cellsto display (third-seventh priority).

Appendix 1.3.2 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is three, there arethree reserved screens, the speaker location has three cameras, thesecond priority location has three cameras, the seventh prioritylocation has two cameras and there are nine additional cells to display(second-seventh priority).

Appendix 1.3.3 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other.

Appendix 1.3.4 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is three, there arethree reserved screens, the speaker location has one camera, the secondand third priority locations have three cameras, and there are sevenadditional cells to display.

Appendix 1.3.5 shows exemplary reserved screens 400 and grid screens 500selected by the layout manager 302 for a first endpoint when the maximumnumber of cameras utilized by the other endpoints is two, there arethree screens, only one reserved, the speaker location has one camera,the second and fourth priority locations have two cameras, and there areseven additional cells to display (second-sixth priority).

Appendix 1.4 shows exemplary grid screens 500 selected by the layoutmanager 302 in the participants priority mode.

Appendix 1.4.1 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is one.

Appendix 1.4.2 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is two.

Appendix 1.4.3 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is three.

Appendix 1.4.4 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is four and at least one of the otherendpoints is utilizing three cameras.

Appendix 1.4.5 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is four and none of the other endpointsare utilizing three cameras.

Appendix 1.5 shows exemplary grid screens 500 selected by the layoutmanager 302 in the speaker priority mode.

Appendix 1.5.1 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is three, the second priority endpointhas three cameras and the total number of cells is ten.

Appendix 1.5.2 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is three, the first and second priorityendpoints have three cameras and the total number of cells is twelve.

Appendix 1.5.3 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is three, the first and second priorityendpoints have three cameras and the total number of cells is ten.

Appendix 1.5.4 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is three, the second and third priorityendpoints have three cameras and the total number of cells is ten.

Appendix 1.5.5 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is two, the second priority endpoint hastwo cameras and the total number of cells is eight.

Appendix 1.5.6 shows exemplary grid screens 500 selected by the layoutmanager 302 for a first endpoint when the maximum number of camerasutilized by the other endpoints is four, the second priority endpointhas four cameras and the total number of cells is eight.

Various modifications, extensions, and changes to the systems andalgorithms described herein may be implemented without departing fromthe spirit and scope of the present invention. Additionally, the variousalgorithms described herein may be implemented in hardware, software,firmware, or any combination thereof.

The invention claimed is:
 1. A videoconferencing device for avideoconferencing system having a plurality of endpoints, the pluralityof endpoints including a first endpoint and one or more additionalendpoints in network communication, each of the plurality of endpointshaving one or more display devices and one or more cameras, thevideoconferencing device comprising: a processor that receives one ormore video streams from each of the plurality of endpoints anddetermines a current speaker endpoint; and a layout manager thatdetermines a spatial arrangement of the video streams output at thefirst endpoint based on a number of display devices being utilized atthe first endpoint and a number of cameras being utilized at each of theone or more additional endpoints, wherein, in a speaker priority mode,the spatial arrangement comprises one or more reserved screens, each ofthe reserved screens comprising one or more primary display areas andone or more secondary display areas that are smaller than the one ormore primary display areas, and the layout manager determines a numberof reserved screens output at the first endpoint based on the number ofdisplay devices being utilized at the first endpoint and the number ofcameras being utilized at each of the one or more additional endpoints.2. The videoconferencing device of claim 1, wherein, in the speakerpriority mode: the layout manager determines the spatial arrangementsuch that the one or more video streams from the current speakerendpoint are output to the one or more primary display areas of at leastone of the one or more reserved screens.
 3. The videoconferencing deviceof claim 1, wherein, in the speaker priority mode, the layout managerdetermines the spatial arrangement such that three reserved screens areoutput at the first endpoint if the first endpoint is utilizing three ormore display devices and at least one of the additional endpoints isutilizing three cameras.
 4. The videoconferencing device of claim 3,wherein, in the speaker priority mode, the layout manager determines thespatial arrangement such that one or more video streams are output froma previous speaker endpoint in the one or more primary display areas ofat least one of the reserved screens if the current speaker endpoint isnot utilizing three cameras.
 5. The videoconferencing device of claim 1,wherein, in the speaker priority mode, the layout manager determines thespatial arrangement such that two reserved screens are output if thefirst endpoint is utilizing two or more display devices, at least oneadditional endpoint is utilizing four cameras, and none of theadditional endpoints are utilizing three cameras.
 6. Thevideoconferencing device of claim 1, wherein, in the speaker prioritymode, the spatial arrangement also comprises one or more grid screenscomprising a plurality of display areas of substantially equal size ifthe number of reserved screens output at the first endpoint is less thanthe number of display devices being utilized at the first endpoint. 7.The videoconferencing device of claim 6, wherein, in the speakerpriority mode: the secondary display areas of the one or more reservedscreens and the display areas of the one or more grid screens arearranged in rows; and the layout manager determines the spatialarrangement such that all video streams from a second endpoint areoutput in the same row if the second endpoint is utilizing multiplecameras.
 8. The videoconferencing device of claim 1, wherein, in aparticipant priority mode, the spatial arrangement comprises one or moregrid screens comprising a plurality of display areas of substantiallyequal size arranged in a plurality of rows and a plurality of columns.9. The videoconferencing device of claim 8, wherein, in the participantpriority mode, the layout manager determines the spatial arrangementsuch that all video streams from a second endpoint are output in thesame row if the second endpoint is utilizing multiple cameras.
 10. Amethod of generating a layout for a videoconference in avideoconferencing system having a plurality of endpoints, the pluralityof endpoints including a first endpoint and one or more additionalendpoints in network communication, each of the plurality of endpointshaving one or more display devices and one or more cameras, the methodcomprising: receiving one or more video streams from each of theplurality of endpoints; determining a current speaker endpoint; anddetermining the layout at the first endpoint based on a number ofdisplay devices being utilized at the first endpoint and a number ofcameras being utilized at each of the one or more additional endpoints,wherein, in a speaker priority mode, the layout comprises one or morereserved screens comprising one or more primary display areas and one ormore secondary display areas that are smaller than the one or moreprimary display areas and determining the layout comprises determining anumber of reserved screens output at the first endpoint based on thenumber of display devices being utilized at the first endpoint and thenumber of cameras being utilized at each of the one or more additionalendpoints.
 11. The method of claim 10, the method further comprising: inthe speaker priority mode, outputting the one or more video streams fromthe current speaker endpoint to the one or more primary display areas ofat least one of the one or more reserved screens.
 12. The method ofclaim 10, wherein, in the speaker priority mode, determining the layoutfurther comprises outputting three reserved screens at the firstendpoint if the first endpoint is utilizing three or more displaydevices and at least one of the additional endpoints is utilizing threecameras.
 13. The method of claim 12, further comprising: in the speakerpriority mode, outputting one or more video streams from a previousspeaker endpoint in the one or more primary display areas of at leastone of the reserved screens if the current speaker endpoint is notutilizing three cameras.
 14. The method of claim 10, wherein, in thespeaker priority mode, determining the layout further comprisesoutputting two reserved screens at the first endpoint if the firstendpoint is utilizing two or more display devices, at least oneadditional endpoint is utilizing four cameras, and none of theadditional endpoints are utilizing three cameras.
 15. The method ofclaim 10, wherein, in the speaker priority mode, the layout furthercomprises one or more grid screens comprising a plurality of displayareas of substantially equal size if the number of reserved screensoutput at the first endpoint is less than the number of display devicesbeing utilized at the first endpoint.
 16. The method of claim 15,wherein, in the speaker priority mode, the secondary display areas ofthe one or more reserved screens and the display areas of the one ormore grid screens are arranged in rows, the method further comprising:outputting all video streams from a second endpoint in the same row ifthe second endpoint is utilizing multiple cameras.
 17. The method ofclaim 10, wherein, in a participant priority mode, the layout comprisesone or more grid screens comprising a plurality of display areas ofsubstantially equal size arranged in a plurality of rows and a pluralityof columns.
 18. The method of claim 17, further comprising: in theparticipant priority mode, outputting all video streams from a secondendpoint in the same row if the second endpoint is utilizing multiplecameras.